Information processing apparatus, information processing method, program, and information processing system

ABSTRACT

There is provided an information processing apparatus including an operation surface configured to receive touch inputs, and a sensor unit configured to detect at least one right-side touch input from a manipulation by a user of the operation surface within a first operational area of the operation surface, and at least one left-side touch input from a manipulation by the user of the operation surface within a second operational area of the operation surface, wherein the first operational area and the second operational area of the operation surface are mapped to a graphical user interface (GUI) area of a display device.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/385,381 (filed on Sep. 15, 2014), which is a National Stage PatentApplication of PCT International Patent Application No.PCT/JP2013/002502 (filed on Apr. 12, 2013) under 35 U.S.C. §371, whichclaims priority to Japanese Patent Application No. 2012-095497 (filed onApr. 19, 2012), which are all hereby incorporated by reference in theirentirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, a program, and an informationprocessing system, and, in particular, to an information processingapparatus, an information processing method, a program, and aninformation processing system which can improve, for example,operability.

BACKGROUND ART

For example, the remote operation of an electronic device, such as aTelevision receiver (TV), is performed by transmitting a command from aremote commander to the TV in such a way as to press a button of theremote commander which is provided with channel buttons and volumebuttons.

Meanwhile, in recent years, in order to support various operations, suchas the operation of a browser on a display screen of the TV and thelike, a remote commander which is provided with a touch sensor fordetecting a touch on an operation surface of a touch pad, a touch panel,or the like has been proposed.

Further, applications have increased which are used to enable a remoteoperation of an electronic device by performing a touch operation usingan apparatus which is provided with a touch sensor like a smart phone.

Operation systems which operate an electronic device other than the TVusing the touch sensor which detects the operation performed by a useron the operation surface can be largely divided into two types, that is,first and second operation systems.

The first operation system is, like the touch pad which is provided on aso-called notebook Personal Computer (PC) or the like, an operationsystem which moves a pointer (a cursor) which is displayed on a displayscreen using, so to speak, a “relative position” operation which is thesame as a mouse operation, and operates a Graphical User Interface (GUI)or the like which is arranged on the display screen.

In the first operation system, a pointer is moved according to arelative position which is touched on the operation surface withreference to a position on the operation surface at which a touch isstarted. Therefore, the pointer is not necessarily displayed at the sameposition even when the same position on the operation surface istouched.

The second operation system is an operation system (for example, referto PTL 1) which associates each point (each position) (coordinates) onthe operation surface of the touch pad or the like which is operated bya user with each point (each position) (coordinates) on the displayscreen of the TV or the like in a one to one correspondence and operatesa pointer which moves according to an “absolute position” on theoperation surface of a touch pad.

In the second operation system, each point on the operation surface hasa one to one correspondence with each point on the display screen.Therefore, if the same position on the operation surface is touched, thepointer is displayed at the same position.

Meanwhile, in order to enable multi-touch which has become popularrecently, the second operation system is necessary to maintain thepositional relationship between respective touch points when theoperation surface is touched by a plurality of fingers.

In the first operation system, a pointer is displayed on the displayscreen regardless of whether a user touches the operation surface of thetouch pad. However, in the second operation system, for example, whenthe user touches the operation surface, a pointer is displayed at aposition on the display screen which corresponds to the touched point(touch point).

Therefore, in the second operation system, when the user releases thetouch from the operation surface (when a finger is taken away), thepointer which has been displayed on the display screen is eliminated.

In the above-described second operation system, for example, it may bepossible to display a pointer on the display screen respectivelyaccording to an operation performed on the operation surface by a fingerof the left hand (a left-hand finger) and an operation performed on theoperation surface by a finger of the right hand (a right-hand finger),and thus it may be possible to perform operation by using both hands.

Therefore, according to the second operation system, when an On ScreenKeyboard (OSK) is operated, it may be possible to rapidly perform theselection of a key, the selection of a predictive candidate, or thelike.

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2009-253773

SUMMARY OF INVENTION Technical Problem

The history of an operation system using a touch sensor is as yet short,and there is room for improvement in operability.

The present disclosure has been made in consideration of at least theabove situation, and it is desirable to enable operability to beimproved.

Solution to Problem

An information processing apparatus according to a first embodiment ofthe present disclosure includes: a mapping unit that performs a mappingwhich associates a position on an operation surface which is operated bya user with a position on a display screen on which an image isdisplayed based on graphics which are displayed on the display screen;and a detection unit that detects a correspondence point which is aposition on the display screen and corresponds to the position on theoperation surface which is operated by the user based on a result of themapping. Furthermore, an information processing program according to thefirst embodiment of the present disclosure may cause a computer tofunction as the information processing apparatus.

An information processing method according to the first embodiment ofthe present disclosure may include: performing a mapping whichassociates a position on an operation surface which is operated by auser with a position on a display screen on which an image is displayedbased on graphics which are displayed on the display screen; anddetecting a correspondence point which is a position on the displayscreen and corresponds to the position on the operation surface which isoperated by the user based on a result of the mapping.

An information processing system according to a second embodiment of thepresent disclosure may include: a terminal that includes the operationsurface which is operated by a user; and an information processingapparatus which performs a process according to an operation of theoperation surface. The information processing apparatus may include amapping unit that performs a mapping which associates a position on theoperation surface with a position on a display screen on which an imageis displayed based on graphics which are displayed on the displayscreen; and a detection unit that detects a correspondence point whichis a position on the display screen and corresponds to the position onthe operation surface which is operated by the user based on a result ofthe mapping.

According to the above-described first and second embodiments, mapping,which associates the position on the operation surface which is operatedby the user with the position on the display screen on which an image isdisplayed, may be performed based on graphics which are displayed on thedisplay screen, and a correspondence point which is a position on thedisplay screen, which corresponds to the position on the operationsurface which is operated by the user, may be detected based on theresult of the mapping.

Meanwhile, the information processing apparatus according to embodimentsmay be an independent apparatus and may be an internal block whichconfigures a single apparatus.

In addition, it is possible to provide the program by transmitting theprogram via a transmission medium or by recording the program on therecording medium.

An information processing apparatus according to further embodiments ofthe present disclosure includes an operation surface configured toreceive touch inputs, and a sensor unit configured to detect at leastone right-side touch input from a manipulation by a user of theoperation surface within a first operational area of the operationsurface, and at least one left-side touch input from a manipulation bythe user of the operation surface within a second operational area ofthe operation surface, wherein the first operational area and the secondoperational area of the operation surface are mapped to a graphical userinterface (GUI) area of a display device.

An information processing method according to further embodiments of thepresent disclosure includes mapping a first operational area and asecond operational area of an operation surface of a terminal device toa graphical user interface (GUI) area of a display device, detecting atleast one right-side touch input from a manipulation by a user of theoperation surface within the first operational area of the operationsurface, and detecting at least one left-side touch input from amanipulation by the user of the operation surface within the secondoperational area of the operation surface.

A non-transitory computer-readable medium according to furtherembodiments of the present disclosure is embodied with a program, whichwhen executed by a computer, causes the computer to perform a methodincluding mapping a first operational area and a second operational areaof an operation surface of a terminal device to a graphical userinterface (GUI) area of a display device, detecting at least oneright-side touch input from a manipulation by a user of the operationsurface within the first operational area of the operation surface, anddetecting at least one left-side touch input from a manipulation by theuser of the operation surface within the second operational area of theoperation surface.

Advantageous Effects of Invention

According to embodiments of the present disclosure, it is possible toimprove the operability.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a perspective view illustrating a configuration example of aninformation processing system in accordance with an embodiment of thepresent disclosure.

FIG. 2 is a perspective view illustrating a configuration example of aterminal in accordance with an embodiment of the present disclosure.

FIG. 3 is a view illustrating the outline of a process of theinformation processing system in accordance with an embodiment of thepresent disclosure.

FIG. 4 is a block diagram illustrating configuration examples of theterminal and a TV in accordance with an embodiment of the presentdisclosure.

FIG. 5 is a flowchart illustrating a process of the terminal inaccordance with an embodiment of the present disclosure.

FIG. 6 is a flowchart illustrating a process of the TV in accordancewith an embodiment of the present disclosure.

FIG. 7 is a view illustrating an example of a mapping which associatesan operation surface 11 with a display screen, in accordance with anembodiment of the present disclosure.

FIG. 8 is a view illustrating a first mapping in accordance with anembodiment of the present disclosure.

FIG. 9 is a view illustrating a second mapping in accordance with anembodiment of the present disclosure.

FIG. 10 is a view illustrating the correspondence relationship between aposition (Xin,Yin) on the operation surface and a correspondence point(Xout,Yout) on the display screen according to the second mapping, inaccordance with an embodiment of the present disclosure.

FIG. 11 is a view illustrating a third mapping in accordance with anembodiment of the present disclosure.

FIG. 12 is a view illustrating the correspondence relationship betweenthe position (Xin,Yin) on the operation surface and the correspondencepoint (Xout,Yout) on the display screen according to the third mapping,in accordance with an embodiment of the present disclosure.

FIG. 13 is a flowchart illustrating a determination process to apply thethird mapping (limited mapping) in accordance with an embodiment of thepresent disclosure.

FIG. 14 is a flowchart illustrating a touch point recognition process inaccordance with an embodiment of the present disclosure.

FIG. 15 is a view illustrating a fourth mapping in accordance with anembodiment of the present disclosure.

FIG. 16 is a view illustrating the correspondence relationship betweenthe position (Xin,Yin) on the operation surface and the correspondencepoint (Xout,Yout) on the display screen according to the fourth mapping,in accordance with an embodiment of the present disclosure.

FIG. 17 is a flowchart illustrating a determination process to apply thefourth mapping (offset mapping) in accordance with an embodiment of thepresent disclosure.

FIG. 18 is a view illustrating a display example of the operationsurface as the touch screen of the terminal in accordance with anembodiment of the present disclosure.

FIG. 19 is a block diagram illustrating a configuration example of acomputer in accordance with an embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

<Information Processing System According to Embodiments>

FIG. 1 is a perspective view illustrating a configuration example of aninformation processing system in accordance with an embodiment of thepresent disclosure.

In FIG. 1, an information processing system includes a terminal 10 and aTV 20.

The terminal 10 is, for example, a tablet terminal such as a smart phoneor the like or a remote commander which performs remote operation on anelectronic device, and functions as the remote commander which performsthe remote operation on the TV 20 in FIG. 1.

The terminal 10 includes an operation surface 11 which has a rectangularshape and is operated by a user, and transmits operation informationwhich indicates the operation performed on the operation surface 11 bythe user to the TV 20.

The TV 20 has a display screen 21 which has a rectangular shape anddisplays an image, and displays the images of content, such as atelevision broadcast program, a program which is recorded by a recorder(not shown), a program which is recorded in a built-in recording medium(not shown), or the like.

In addition, the TV 20 displays a Graphic User Interface (GUI), that is,for example, graphics of a browser or the like, on the display screen21.

Further, the TV 20 receives operation information which is transmittedfrom the terminal 10, and performs a process, such as selection of achannel (e.g., channel tuning), increase or decrease in volume, displayof a menu, a process corresponding to a GUI such as the browser of thelike, in accordance with the operation information.

In the information processing system which is configured as describedabove, the operation information which includes information indicativeof a touch point, such as the coordinates of a position (e.g., a touchpoint) on the operation surface 11 which is touched by a user in theterminal 10, is transmitted to the TV 20.

In the TV 20, a correspondence point which is a position on the displayscreen 21 corresponding to the touch point is detected based on theoperation information from the terminal 10, and a process according tothe correspondence point is performed.

FIG. 2 is a perspective view illustrating a configuration example of aterminal 10 in FIG. 1, in accordance with an embodiment of the presentdisclosure.

The terminal 10 has a planar shape, and is provided with the operationsurface 11, which is operated by the user and has a rectangular shape,on one surface thereof.

Here, a surface of the terminal 10 on which the operation surface 11 isprovided is referred to as a front surface (e.g., superficial surface)of the terminal 10.

In addition, when the longer direction of the operation surface 11 whichhas the rectangular shape is horizontally arranged, one of two longsides in a rectangle as the operation surface 11 is referred to as anupper side of the terminal 10.

The terminal 10 includes, for example, a built-in set of anelectrostatic (e.g., electrostatic capacitance) type touch panel or atouch pad, a light source which emits light, and a sensor which receivesreflection light from an object, and has a function of receiving (e.g.,detecting) input (e.g., approach and contact) to the operation surface11 from the outside.

If an object from the outside, that is, for example, a finger of theuser or a touch pen which is used by the user approaches or comes intocontact with (e.g., touches) the operation surface 11, the terminal 10transmits the operation information which includes positionalinformation indicative of the position approached or contacted.

Hereinafter, although the description primarily discusses the cominginto contact by the finger of the user or a touch pen with the operationsurface 11, embodiments of the disclosure also contemplate the approachby the finger of the user or a touch pen to the operation surface 11 inaddition to or in lieu thereof.

As the positional information indicative of a position (e.g., touchpoint) with which a finger of the user comes into contact, it ispossible to use information which can specify a touch point on theoperation surface 11 with which a finger comes into contact, forexample, such as coordinates of the touch point on the operation surface11, information indicative of the degree of contact with regard to eachpoint of the operation surface 11 (for example, distribution ofelectrostatic capacitance of the operation surface 11 when the operationsurface 11 is configured by an electrostatic touch panel or the like).

In addition, in the terminal 10, as a coordinate system of thepositional information indicative of the touch point, it is possible touse, for example, a fixed 2-dimensional coordinate system in which adirection upward from the terminal 10 (indicated using a white-spacedarrow in FIG. 2) is set to a y axis, the vertical direction of theoperation surface 11 is set to an x axis, and a predetermined point on a2-dimensional plane which includes the operation surface 11 is set tothe origin.

Further, in the terminal 10, it is possible to detect the posture of theterminal 10 when the user holds the terminal 10, and to use a variable2-dimensional coordinate system according to the posture of the terminal10 as the coordinate system of the positional information of the touchpoint, based on, for example, the shadow of a finger of the user on theoperation surface 11 (e.g., shadow which appears when a finger of theuser is vertically projected on the operation surface 11), a gravitysensor, or the like.

That is, in the terminal 10, it is possible to use the variable2-dimensional coordinate system which varies according to the posture ofthe terminal 10, in which the vertical direction of the operationsurface 11 is set to a y axis, the horizontal direction thereof is setto an x axis, and a predetermined point on the operation surface 11 isset to the origin when the user holds the terminal 10, as the coordinatesystem of the positional information of the touch point.

Meanwhile, as described above, in the TV 20, the correspondence point onthe display screen 21 which corresponds to the touch point is detectedbased on the operation information from the terminal 10. However, whenthe size of the operation surface 11 (for example, the minimum value andthe maximum value of each of an x-coordinate and a y-coordinate) isnecessary in addition to the positional information in order to detectthe correspondence point, the operation information which is transmittedfrom the terminal 10 to the TV 20 also includes information, such as thesize of the operation surface 11, which is necessary to detect thecorrespondence point, in addition to the positional information ifnecessary.

Hereinafter, for ease of explanation given as an example, if nototherwise specified, the user causes the opposite side (e.g., lowerside) of the upper side of the terminal 10 to face a position which isnear the user, and operates the operation surface 11 using a finger of aright or left hand in a state in which the end portion of the terminal10 (e.g., the operation surface 11) in the longer direction is graspedby the right or left hand (hereinafter, called a “standard state”) suchthat the user can see the operation surface 11. However, the presentdisclosure is not limited thereto.

The operation which is performed on the terminal 10 by the user largelyincludes an operation (e.g., pointing) which is performed to select abutton or the like on the GUI displayed on the display screen 21 of theTV 20, and a determination operation which is performed to determine(e.g. confirm) the selection.

As shown in FIG. 2, physical buttons 13 are provided on the frontsurface, the rear surface (e.g., of the opposite side surface to thefront surface), and the side surface of the terminal 10 if necessary.Therefore, in the terminal 10, it is possible to allocate, for example,the operations of the buttons 13 which are provided on the side surfaceof the terminal 10 to the determination operation.

That is, for example, when the user holds the terminal 10 in thestandard state, it is possible to easily operate switches 13 which areprovided on the left side surface of the upper side of the terminal 10using an index finger or the like of the left hand, and it is possibleto easily operate the switches 13 which are provided on the right sidesurface of the upper side of the terminal 10 using a right-hand indexfinger.

Here, it is possible to allocate the operation of the switches 13 whichare provided on the left side surface of the upper side of the terminal10 to the determination operation which is performed to determine aselection performed by the operation on the operation surface 11 using afinger of a left hand. In the same manner, it is possible to allocatethe operation of the switches 13 which are provided on the right sidesurface of the upper side of the terminal 10 to the determinationoperation which is performed to determine the selection performed by theoperation on the operation surface 11 using a finger of a right hand.

In addition, in the terminal 10, for example, it is possible to allocatea release operation which takes away a finger which is in contact withthe operation surface 11 from the operation surface 11 to adetermination operation.

That is, it is possible to allocate an operation, which takes away aleft-hand finger which is in contact with the operation surface 11, to adetermination operation in order to determine a selection which isperformed, immediately therebefore, by an operation (e.g., contact) onthe operation surface 11 using a left-hand finger. In the same manner,it is possible to allocate the operation, which takes away a right-handfinger which is in contact with the operation surface 11, to adetermination operation in order to determine a selection which isperformed, immediately therebefore, by an operation on the operationsurface 11 using a right-hand finger.

Further, in the terminal 10, for example, it is possible to allocate apressing operation which presses the operation surface 11 using a fingerwhich comes into contact with the operation surface 11 to thedetermination operation.

In the terminal 10, with regard to the pressing operation performed onthe operation surface 11, for example, as shown in FIG. 2, pressure(e.g., load) sensors which detect a pressure pressing the operationsurface 11 are provided on the respective end portions of the right andleft (e.g., the longer direction) of the operation surface 11, and it ispossible to detect whether or not the pressing operation is performedusing either a left-hand finger or a right-hand finger of the user basedon pressures detected by the respective pressure sensors.

More specifically, it is possible to calculate the pressing power of therespective touch points of a left-hand finger and a right-hand finger ofthe user using the pressures which are output from the respective rightand left pressure sensors on the operation surface 11 and (e.g., thecoordinates of) the touch points of a left-hand finger and a right-handfinger on the operation surface 11, and it is possible to determinewhether or not the pressing operation is performed on a touch point, atwhich the pressing power is greater than a predetermined thresholdvalue, as the determination operation. Otherwise, it is possible todetermine whether or not the pressing operation is performed on a touchpoint of a left-hand finger or a right-hand finger of the user as thedetermination operation, based on, for example, a ratio of pressureswhich are output from the respective right and left pressure sensors onthe operation surface 11.

Meanwhile, the positions at which the pressure sensors are provided arenot limited to the right and left end portions of the operation surface11. Alternatively, only a single pressure sensor may be provided, orthree or more pressure sensors may be provided.

In addition, in the terminal 10, with regard to the pressing operationperformed on the operation surface 11, for example, the operationsurface 11 is configured such that the operation surface 11 can bepressed (e.g., the inside thereof can be pressed), and click switcheswhich detect that the operation surface 11 is pressed are provided onthe right and left end portions of the operation surface 11. It ispossible to detect whether or not the pressing operation is performedusing either a left-hand finger or a right-hand finger of the user basedon the on and off states of each of the click switches.

In addition, in the terminal 10, with regard to the pressing operationperformed on the operation surface 11, for example, the operationsurface 11 is configured such that the operation surface 11 can bepressed, the operation surface 11 is configured with an electrostatictouch-pad or the like, and a single click switch which detects the pressof the operation surface 11 is provided. It is possible to detectwhether or not the pressing operation is performed using either aleft-hand finger or a right-hand finger of the user based on variationsin the electrostatic capacitance of the operation surface 11 when theclick switch is in an on state.

Further, in the terminal 10, with regard to the pressing operationperformed on the operation surface 11, for example, the operationsurface 11 is configured with an electrostatic touch-pad or the like. Itis possible to detect the press of the operation surface 11 and a factthat the press is performed by an operation using any of a left-handfinger and a right-hand finger of the user based on variation in theelectrostatic capacitance of the operation surface 11.

In addition, when it is possible to operate the operation surface 11 bythe approach of a finger or the like of the user, it is possible toallocate the contact with (e.g., touch on) the operation surface to thedetermination operation.

FIG. 3 is a view illustrating the outline of a process of theinformation processing system in FIG. 1, in accordance with anembodiment of the present disclosure.

In FIG. 3, in the TV 20, the image of a program (e.g., a program image)is displayed on a display screen 21, and, further, an icon 31 whichconfigures the GUI is overlapped with the program image and displayed.

Here, the icon 31 functions as various types of buttons (e.g., virtualbuttons) such as a volume button which adjusts volume, a music selectionbutton which selects a channel, and the like.

The terminal 10, for example, regularly, or irregularly, when theoperation state of the terminal 10 is varied or the like, transmits theoperation information to the TV 20.

The TV 20 recognizes (e.g., detects) the operation state of the terminal10 based on the operation information from the terminal 10, and when aleft-hand finger of the user touches the operation surface 11 of theterminal 10, the TV 20 displays a pointer 30L as a symbol image whichindicates that a left-hand finger touches a correspondence point(hereinafter, referred to as a left correspondence point) on the displayscreen 21 which corresponds to the touch point (hereinafter, referred toas a left touch point) of the operation surface 11 which is touched bythe left-hand finger.

In a similar manner, when the operation surface 11 of the terminal 10 istouched by the right-hand finger of the user, the TV 20 displays apointer 30R as a symbol image which indicates that the right-hand fingertouches a correspondence point (hereinafter, referred to as a rightcorrespondence point) on the display screen 21 which corresponds to thetouch point (hereinafter, referred to as a right touch point) of theoperation surface 11 which is touched by the right-hand finger.

It is possible to display the pointers 30L and 30R such that the usereasily distinguishes the respective pointers in such a way as to changea display format such as a color, a pattern, a shape, or the like.

In addition, in FIG. 3, circular graphics are used as a pointer 30Lwhich indicates (e.g., the touch of) a left-hand finger. However, it ispossible to use, for example, an arrow-shaped graphics, graphics in ashape of the left-hand finger which is in contact with the operationsurface 11, graphics in the shape of the left-hand finger whichexpresses the degree of contact (e.g., approach) of the left-hand fingerwith the operation surface 11 using shades or colors, or the like, asthe pointer 30L in addition to the circular graphics. The pointer 30Rwhich indicates (e.g., the touch of) the right-hand finger, may berepresented similarly.

If the user brings a finger (e.g., a left-hand finger or a right-handfinger) of the user into contact with the operation surface 11, thepointer 30L or 30R is displayed at a correspondence point on the displayscreen 21 which corresponds to the touch point on the operation surface11 which is touched by a finger of the user.

When the user moves a finger which touches on the operation surface 11while touching on the operation surface 11, the pointer 30L or 30R movesaccording to the movement of the finger of the user. In FIG. 3, thetouch points on the operation surface 11 are in one to onecorrespondence with the correspondence points on the display screen 21,and the user can perform an operation at an interval which might be incontact with the display screen 21.

When the pointer 30L or 30R is moved, for example, onto a desired icon31 in such a way that the user moves the finger which touches on theoperation surface 11 and the determination operation is performed in astate in which the pointer 30L or 30R is positioned on the icon 31(e.g., is pointed), the terminal 10 transmits operation informationindicative of the determination operation performed by the user to theTV 20.

When the TV 20 receives the operation information indicative of thedetermination operation, the TV 20 performs a process corresponding tothe icon 31 on which the pointer 30L or 30R is positioned when thedetermination operation is performed, that is, for example, theadjustment of a volume or the selection of a channel.

<Configuration Examples of Terminal 10 and TV 20>

FIG. 4 is a block diagram illustrating configuration examples of aterminal 10 and a TV 20 in FIG. 1, in accordance with an embodiment ofthe present disclosure.

In FIG. 4, the terminal 10 includes an operation surface 11, a sensor12, a button group 13, a control unit 41, a memory 42, and acommunication unit 43.

The operation surface 11 is, for example, a rectangular-shaped surfacewhich is operated by a user. The sensor 12 detects an operationperformed on the operation surface 11, and supplies detectioninformation indicative of the result of detection to the control unit41.

Meanwhile, the sensor 12 includes a necessary sensor such as thepressure sensor described in FIG. 2 when necessary in addition to atouch sensor such as a touch-pad or a touch panel.

In addition, the operation surface 11 is a part of the touch sensor.

Further, when a touch panel which has a transparent operation surface 11is used as the touch sensor, it is possible to integrally configure theoperation surface 11 and a display unit, such as a liquid crystal panelor the like, in a single body. In this case, the operation surface 11displays an image, and functions as a touch screen which can receive aninput for the image. A necessary image, such as a GUI or the like, isdisplayed on the display unit which configures the touch screen, underthe control of the control unit 41.

The button 13 is a physical button described in FIG. 2 which is providedon the side of the terminal 10, and operated by the user.

The control unit 41 controls each block which configures the terminal10, and performs other necessary processes.

That is, the control unit 41 generates operation information indicativeof an operation performed by the user with respect to the terminal 10which includes the operation surface 11, based on the detectioninformation from the sensor 12 or the operation of the button 13, andsupplies the generated operation information to the communication unit43. In addition, the control unit 41 performs a necessary processaccording to the information which is received by the communication unit43.

The memory 42 stores data (e.g., information) which is necessary for theoperation of the control unit 41.

The communication unit 43 performs communication in a wireless manner(e.g., wireless communication), for example, such as wireless Local AreaNetwork (LAN), IEEE 802.15 (so-called Bluetooth (registered trademark)),infrared rays, or the like.

That is, the communication unit 43 transmits the operation informationfrom the control unit 41 to the TV 51.

In addition, the communication unit 43 receives information which iswirelessly transmitted from the TV 20 or the like, and supplies theinformation to the control unit 41.

The TV 20 includes a communication unit 51, an information processingunit 52, a display unit 53, and a TV function unit 54.

The communication unit 51 performs wireless communication like thecommunication unit 43.

That is, the communication unit 51 transmits information which issupplied from the information processing unit 52 or the like. Inaddition, the communication unit 51 receives the operation informationwhich is transmitted from (e.g., the communication unit 43 of) theterminal 10, and supplies the operation information to the informationprocessing unit 52.

The information processing unit 52 performs various types of information(e.g., signal) processes in order to remotely operate the TV 20 with theterminal 10 using the operation information which is supplied from thecommunication unit 51.

That is, the information processing unit 52 includes a display controlunit 61, a mapping unit 62, and a correspondence point detection 63.

The display control unit 61 performs display control which displays animage on the display unit 53.

That is, the display control unit 61, for example, generates andacquires a GUI such as a browser, an OSK, or the like, and displays theGUI on the display unit 53. In addition, the display control unit 61,for example, displays the pointer 30L or 30R on the display unit 53 as asymbol image, which indicates that the operation surface 11 of theterminal 10 is touched, at a correspondence point on the display screen21 of the display unit 53 which is detected by the correspondence pointdetection unit 63.

The mapping unit 62 performs a mapping which associates a position onthe operation surface 11 of the terminal 10 with a position on thedisplay screen 21 of the display unit 53, based on a GUI which isdisplayed on the display unit 53.

The correspondence point detection unit 63 detects a correspondencepoint on the display screen 21 of the display unit 53 which correspondsto the touch point on the operation surface 11 which is touched by theuser, and which is recognized using the operation information from thecommunication unit 51, based on the result of mapping performed by themapping unit 62.

The display unit 53 is, for example, a display panel which includes thedisplay screen 21 such as a liquid crystal panel, an organic ElectroLuminescence (EL) panel, or the like, and displays the GUI, the pointer30L or 30R, or the like on the display screen 21 under the control ofthe display control unit 61.

In addition, the display unit 53 displays an image which is suppliedfrom the TV function unit 54.

The TV function unit 54 is a program which causes the TV 20 to functionas a TV, and performs, for example, the reception of a televisionbroadcast program, the recording of the content of a televisionbroadcast program or the like, the reproduction of the recorded content,or the like.

The television broadcast program which is received using the TV functionunit 54 or the image of the reproduced content is supplied from the TVfunction unit 54 to the display unit 53, and displayed on the displayscreen 21.

FIG. 5 is a flowchart illustrating a process of the terminal 10 in FIG.4, in accordance with an embodiment of the present disclosure.

The control unit 41 generates and acquires the operation informationaccording to the output of the sensor 12 or the operation of the button13, and supplies the operation information to the communication unit 43in step S11. The process proceeds to step S12.

In step S12, the communication unit 43 transmits the operationinformation from the control unit 43. Thereafter, the process returnsfrom step S12 to step S11. Hereinafter, the same process is repeated.

FIG. 6 is a flowchart illustrating a process of the TV 20 in FIG. 4, inaccordance with an embodiment of the present disclosure.

In step S21, the display control unit 61 generates a GUI according to anevent other than the operation performed by the user, supplies the GUIto the display unit 53, and displays the GUI on the display screen 21.The process proceeds to step S22.

In step S22, the mapping unit 62 performs a mapping which associates aposition on the operation surface 11 of the terminal 10 with a positionon the display screen 21 of the display unit 53 based on the GUI whichis displayed on the display screen 21. The process proceeds to step S23.

In step S23, the communication unit 51 waits for the operationinformation to be transmitted from the terminal 10, receives theoperation information, and supplies the operation information to theinformation processing unit 52. The process proceeds to step S24.

In step S24, the correspondence point detection unit 63 of theinformation processing unit 52 detects the correspondence point on thedisplay screen 21 of the display unit 53, which is recognized using theoperation information from the communication unit 51 and whichcorresponds to the touch point on the operation surface 11 touched bythe user, based on the result of mapping which is performed immediatelybefore by mapping unit 62. The process proceeds to step S25.

In step S25, the display control unit 61 displays the pointer 30L or 30Rat the correspondence point on the display screen 21 of the display unit53 which is detected by the correspondence point detection unit 63. Theprocess returns to step S23.

<Display Example of Display Screen 21>

FIG. 7 is a view illustrating an example of a mapping which associates(a position on) the operation surface 11 with (a position on) thedisplay screen 21, in accordance with an embodiment of the presentdisclosure.

In FIG. 7, the mapping is performed such that the whole operationsurface 11 equally corresponds to the whole display screen 21.Hereinafter, the mapping is appropriately referred to as a normalmapping.

Here, if it is assumed that the length H2 of the horizontal direction(e.g., lateral direction) of the display screen 21 is kx (=H2/H1) timesas large as the length H1 of the horizontal direction (e.g., longerdirection) of the operation surface 11 and that the length V2 of thevertical direction (e.g., longitudinal direction) of the display screen21 is ky (=V2/V1) times as large as the length V1 of the verticaldirection of the operation surface 11, points (kx×Xin,ky×Yin) areassociated with touch points (Xin,Yin) (e.g., a point and/or position inwhich an x-coordinate is Xin and a y-coordinate is Yin) as acorrespondence point (Xout,Yout) in the normal mapping.

Meanwhile, here, for brief explanation, for example, it assumed that theorigin of the 2-dimensional coordinate system which defines the touchpoints (Xin,Yin) on the operation surface 11 is a lower left point(e.g., vertex) of the operation surface 11 and that the origin of the2-dimensional coordinate system which defines the correspondence points(Xout,Yout) on the display screen 21 is the lower left point of thedisplay screen 21.

FIG. 7 displays an input section which is used to input letter stringswhich are searched for on the web, a predictive candidate section inwhich the candidates of the letter strings, which are predicted to beinput based on the letter (e.g., string) input to the input section, aredisplayed, and a GUI which uses an OSK as a GUI component (hereinafter,referred to as a search GUI).

In addition, in FIG. 7, the search GUI is displayed in a form in whichthe search GUI is overlapped with the image (e.g., the program image) ofa television broadcast program.

Here, in FIG. 7, the predictive candidates are displayed in thepredictive candidate section in such a way that the predictivecandidates are divided into two columns on a left side and a right side.The left side predictive candidates are letter string candidates whichare predicted to be input based on the letter (e.g., string) input inthe input section. The right side predictive candidates are thecandidate titles of content recorded in the TV 20, which are predictedto be searched for based on the letter (e.g., string) input in the inputsection.

Incidentally, in FIG. 7, the search GUI is semitransparent such that theprogram image is visible as much as possible. Further, in FIG. 7, thevertical length of the search GUI approximately coincides with thevertical length of the display screen 21. However, the horizontal widthof the search GUI is shorter than the horizontal width of the displayscreen 21 by a certain margin such that the program image is visible asmuch as possible (such that the user easily confirms the program imageas much as possible).

In addition, the search GUI is displayed in the center of the displayscreen 21, and thus the program image with which the search GUI isoverlapped is displayed in a state in which the right and left portionsthereof are clearly viewed.

Hereinafter, first to fourth mappings will be described as mappingswhich can improve the operability of the terminal 10 compared to theabove-described normal mapping.

<First Mapping>

FIG. 8 is a view illustrating a first mapping which associates theoperation surface 11 with the display screen 21, in accordance with anembodiment of the present disclosure.

In the normal mapping, regardless of the GUI which is displayed on thedisplay screen 21, for example, the whole operation surface 11 isuniformly associated with the whole display screen 21 as described withreference to FIG. 7.

Therefore, in the normal mapping, in addition to the region of thedisplay screen 21, on which the GUI is displayed (hereinafter, referredto as a GUI region), the user can operate (e.g., point) regions otherthan the GUI region (hereinafter, referred to as a non-GUI region).

Here, “operating a region on the display screen 21” means operating theoperation surface 11 such that a correspondence point is positioned inthe region.

As described above, in the normal mapping, the user can operate thenon-GUI region. However, since the GUI is not displayed in the non-GUIregion, it is not necessary to operate the non-GUI region.

Here, in the first mapping, the mapping unit 62 associates positions onthe GUI regions (e.g., graphic regions) which display the GUIs in aposition on the operation surface 11 and a position on the displayscreen 21, based on the GUI which is displayed on the display screen 21.

In FIG. 8, the mapping is performed such that the whole operationsurface 11 is uniformly associated with the whole GUI region.

Here, the mapping which uniformly associates the whole (or a part of)operation surface 11 with the whole GUI region is referred to as auniform mapping.

In the uniform mapping, if it is assumed that the length H2 of thehorizontal direction of the GUI region is kx (=H2/H1) times as large asthe length H1 of the horizontal direction of the operation surface 11and that the length V2 of the vertical direction of the GUI region is ky(=V2/V1) times as large as the length V1 of the vertical direction ofthe operation surface 11, a position (Xin,Yin) on the operation surface11 is associated with a position (kx×Xin,ky×Yin) on the GUI region.

Meanwhile, here, for brief explanation, for example, it is assumed thatthe GUI region is a rectangular-shaped region, the origin of the2-dimensional coordinate system which defines positions (e.g., points)(Xin,Yin) on the operation surface 11 is the lower left point (e.g.,vertex) of the operation surface 11, and the origin of the 2-dimensionalcoordinate system which defines positions (Xout,Yout) on the GUI regionis the lower left point of the GUI region.

The GUI region is equal in size to the display screen 21 in its largestembodiment, and is often configured to be smaller than the displayscreen 21.

In FIG. 8, the search GUI which is described with reference to FIG. 7 isdisplayed. The search GUI is smaller than the display screen 21 asdescribed with reference to FIG. 7.

Here, it is assumed that a region which has an arbitrary size of thedisplay screen 21 is a unit region. According to the uniform mapping(e.g., the first mapping) which associates the whole operation surface11 with the whole GUI region, a wider range on the operation surface 11is associated with the unit region compared to the normal mapping whichassociates the whole operation surface 11 with the whole display screen21.

That is, for example, as shown in FIG. 8, when an OSK is displayed inthe display region 21, a wider range of the operation surface 11 isassociated with (e.g., the display region of) each of the keys of theOSK according to the uniform mapping compared to the normal mapping.

As the result, compared to the case of the normal mapping, the user canoperate the operation surface 11 with comparatively rough precisionwhile sensing that each of the keys of the OSK becomes large, and thusit is possible to improve the operability of the terminal 10 (i.e., theoperability of the OSK).

Here, in the uniform mapping, it is possible to reduce a region on thedisplay screen 21 which can be set to be a correspondence point to theGUI region from the whole display screen 21, compared to the case of thenormal mapping, according to the GUI displayed on the display screen 21.

Meanwhile, when a large GUI which is unlikely to be displayed on thedisplay screen 21 is displayed, as a result, it is possible to associatethe whole operation surface 11 with the whole display screen 21 in theuniform mapping in the same manner as the normal mapping. In additionthereto, it is possible to assume a virtual display screen in which thedisplay screen 21 is enlarged such that the large GUI can be displayed,and to associate a GUI region obtained when the whole of the large GUIis displayed with the operation surface 11 on the virtual displayscreen.

In the uniform mapping, the GUI region, obtained when the whole of thelarge GUI is displayed on the virtual screen in which the display screen21 is enlarged, is associated with the operation surface 11. Then, inthe uniform mapping, compared to the case of the normal mapping, it ispossible to enlarge a region on the display screen 21 which can be acorrespondence point to a large GUI region which is displayed on thevirtual display screen, according to the GUI which is displayed on thedisplay screen 21.

Meanwhile, when a plurality of GUIs is displayed on the display screen21, the mapping unit 62 performs a mapping based on an activated (e.g.,focused) GUI. Therefore, if the activated GUI is changed, the mappingunit 62 performs a mapping again based on the changed GUI.

<Second Mapping>

FIG. 9 is a view illustrating a second mapping which associates theoperation surface 11 with the display screen 21, in accordance with anembodiment of the present disclosure.

The second mapping associates a position on the operation surface 11with a position on the GUI region based on GUI components whichconfigures the GUI which are displayed on the display screen 21.Therefore, a region on which a GUI component having a small (or large)size is displayed is associated with a wide (or narrow) region of theoperation surface 11 compared to the case of the uniform mapping.

Here, in FIG. 9, on the display screen 21, the search GUI which uses theinput section, the predictive candidate section, and the OSK as GUIcomponents, is displayed as described with reference to FIG. 7.

Meanwhile, the predictive candidate section which is the GUI componentis further configured such that each of the predictive candidates isused as a GUI component. In addition, the OSK which is a GUI componentis further configured such that each key is used as a GUI component.

In the search GUI, for example, when attention is paid to the predictivecandidate section and the OSK, a predictive candidate which is displayedin the predictive candidate section has a larger horizontal width thaneach of the keys of the OSK. Therefore, the degree of difficulty of theoperation of the predictive candidate is relatively low (i.e., it iseasy to be operated) and the degree of difficulty of the operation ofthe key of the OSK is high (i.e., it is difficult to be operated).

Here, in the second mapping, based on the GUI components which configurethe GUI displayed on the display screen 21, a region in which a GUIcomponent having a small size is associated with a wide range of theoperation surface 11 compared to the uniform mapping.

That is, in FIG. 9, in the second mapping, a GUI component is associatedwith the range of the operation surface 11 having a vertical width whichis inversely proportional to the horizontal width of the GUI component.

Therefore, in FIG. 9, a predictive candidate having a wide horizontalwidth is associated with the range of the operation surface 11 having anarrow vertical width compared to the case of the uniform mapping, andthe key of the OSK having a narrow horizontal width is associated withthe range of the operation surface 11 having a wide vertical widthcompared to the case of the uniform mapping.

FIG. 10 is a view illustrating the correspondence relationship between aposition (Xin,Yin) on the operation surface 11 and a correspondencepoint (Xout,Yout) on the display screen 21 according to the secondmapping, in accordance with an embodiment of the present disclosure.

Meanwhile, it is assumed that the length H2 of the horizontal directionof the GUI region is kx (=H2/H1) times as large as the length H1 of thehorizontal direction of the operation surface 11, and that the length V2of the vertical direction of the GUI region is ky (=V2/V1) times aslarge as the length V1 of the vertical direction of the operationsurface 11. Further, it is assumed that kx=ky=k.

In addition, for brief explanation, it is assumed that the GUI region isa rectangular-shaped region, that the origin of the 2-dimensionalcoordinate system which defines positions (Xin,Yin) on the operationsurface 11 is the lower left point (vertex) of the operation surface 11,and that the GUI region of the 2-dimensional coordinate system whichdefines positions (Xout,Yout) is the lower left point on the GUI region.

In the uniform mapping, as shown in FIG. 10 using a dashed line, thepositions (Xin,Yin) on the operation surface 11 is associated with thepositions (k×Xin,k×Yin) on the GUI region.

Meanwhile, in FIG. 10, the correspondence relationship of thex-coordinate is omitted and only the correspondence relationship of they-coordinate is shown.

In the second mapping, as shown in FIG. 10 using a solid line, theposition Yin on the operation surface 11 (e.g., a position where they-coordinate is positioned on the operation surface 11 of Yin) isassociated with a position Yout=f(Yin) on the GUI region.

Here, the function f(Yin) indicates the second mapping.

According to the function f(Yin) in FIG. 10, the OSK (e.g., a region inwhich the OSK is displayed) is associated with the range of theoperation surface 11 which has a wider vertical width (e.g., a width inthe y direction) than that in the case of the uniform mapping, thepredictive candidate section is associated with the range of theoperation surface 11 having a narrower vertical width (e.g., a width inthe y direction) than the case of the uniform mapping, and the inputsection is associated with the range of the operation surface 11 havingthe same vertical width as the case of the uniform mapping,respectively.

Meanwhile, in the second mapping in FIG. 9, the same mapping as theuniform mapping is performed on the horizontal direction.

As described above, according to the second mapping, since the OSK whichhas a narrow horizontal width of a key is associated with the range ofthe operation surface 11 having a wider vertical width than that in thecase of the uniform mapping, the user who operates the operation surface11 can operate the OSK while sensing that the vertical width of each ofthe keys of the OSK is practically widened, and thus it is possible toimprove the operability of the OSK by lowering the degree of difficultyin operating the keys of the OSK.

That is, according to the second mapping, for example, it is possible toeasily point to a GUI component, such as the key of the OSK, a linkbutton to a web page, or the like, which is small in size, and which hasa high degree of difficulty in an operation which asks for an elaboratefinger movement in the uniform mapping.

Meanwhile, according to the second mapping in FIGS. 9 and 10, since thepredictive candidate section is associated with the range of theoperation surface 11 having a narrower vertical width than that in theuniform mapping, the degree of difficulty of the operation of thepredictive candidate in the predictive candidate section increases inthe vertical direction (e.g., y direction).

However, the predictive candidate has a wide horizontal width andoperability which is the same as the case of the uniform mapping issecured in the horizontal direction. Therefore, generally, there islittle influence on the operability of the predictive candidate section,obtained by associating the predictive candidate section with the rangeof the operation surface 11 having a narrower vertical width than thatin the case of the uniform mapping.

Here, in the second mapping, it is possible to correct a correspondencepoint in the case of the normal mapping, which corresponds to theposition on the operation surface 11, according to (e.g., the size) ofthe GUI component which is displayed on the display screen 21.

Meanwhile, in FIGS. 9 and 10, as the second mapping, a GUI component isassociated with the range of the operation surface 11 having a verticalwidth which is inversely proportional to the horizontal width of the GUIcomponent (e.g., the range in the vertical width becomes larger (orsmaller) as the horizontal width of the GUI component becomes smaller(or larger)). However, in addition thereto, in the second mapping, theGUI component can be associated with the range of the operation surface11 having a horizontal width which is inversely proportional to thevertical width of the GUI component.

In addition, in FIGS. 9 and 10, in the second mapping, a position on theGUI region is associated with a position on the operation surface 11 ora position on the display screen 21 in the same manner as the firstmapping. However, in the second mapping, in addition thereto, forexample, it is possible to associate the whole operation surface 11 withthe whole display screen 21 in the same manner as the normal mapping.

<Third Mapping>

FIG. 11 is a view illustrating a third mapping which associates theoperation surface 11 with the display screen 21, in accordance with anembodiment of the present disclosure.

In the first and second mappings, the same mapping is applied regardlessof whether a touch point on the operation surface 11 is a left touchpoint which is touched by a left-hand finger of the user or a righttouch point which is touched by a right-hand finger of the user.However, in the third mapping, different types of mappings are performedon the left touch point and the right touch point (e.g., an operationperformed on the operation surface 11 by the left hand of the user andan operation performed on the operation surface 11 by the right hand).

That is, in the third mapping, with regard to the left touch point, apart of the operation surface 11 is associated with a part of the GUIregion. With regard to the right touch point, another part of theoperation surface 11 is associated with a remaining part of the GUIregion.

In FIG. 11, with regard to the left touch point, the approximately lefthalf range of the operation surface 11 (hereinafter, referred to as aleft operation range) as a part of the left side of the operationsurface 11 is uniformly associated with the left half of the GUI region.With regard to the right touch point, the approximately right half rangeof the operation surface 11 (hereinafter, referred to as a rightoperation range) as a part of the right side of the operation surface 11is uniformly associated with the right half of the GUI region.

In addition, in FIG. 11, there is a gap between the left operation rangeand the right operation range of the operation surface 11.

Further, in FIG. 11, the boundary on the right side of the left halfregion of the GUI region (hereinafter, referred to as a left GUI region)which is associated with the left operation range of the operationsurface 11 matches with the boundaries on the right sides of “T”, “G”,and “V” keys which are arranged on the straight line of the OSK.

In addition, in FIG. 11, the boundary on the left side of the right halfregion of the GUI region (hereinafter, referred to as a right GUIregion) which is associated with the right operation range of theoperation surface 11 matches with the boundaries on the left sides of“Y”, “H”, and “B” keys which are arranged on the straight line of theOSK.

FIG. 12 is a view illustrating the correspondence relationship betweenthe positions (Xin,Yin) on the operation surface 11 and thecorrespondence points (Xout,Yout) on the display screen 21 obtained byperforming the third mapping, in accordance with an embodiment of thepresent disclosure.

Meanwhile, it is assumed that the length H2 of the horizontal directionof the GUI region is kx (=H2/H1) times as large as the length H1 of thehorizontal direction of the operation surface 11 and that the length V2of the vertical direction of the GUI region is ky (=V2/V1) times aslarge as the length V1 of the vertical direction of the operationsurface 11. Further, it is assumed that kx=ky=k.

In addition, for brief explanation, it is assumed that the GUI region isa rectangular-shaped region, the lower left point (e.g., positions) onthe operation surface 11 are points (Xin_min,Yin_min), and the upperright points are points (Xin_max,Yin_max). Further, it is assumed thatthe lower left points on the GUI region are points (Xout_min,Yout_min)and upper right points are points (Xout_max,Yout_max).

For example, if it is assumed that(Xin_min,Yin_min)=(Xout_min,Yout_min)=(0,0) at present, in the uniformmapping, the positions (Xin,Yin) on the operation surface 11 areassociated with the positions (k×Xin,k×Yin) on the GUI region as shownin FIG. 10 using the dashed line.

Meanwhile, in FIG. 12, the correspondence relationship of they-coordinate is omitted and only the correspondence relationship of thex-coordinate is shown.

In the third mapping, with regard to a left touch point Xin_L on theoperation surface 11 (e.g., a left touch point on the operation surface11 in which the x-coordinate is Xin_L), the left touch point Xin_L isassociated with a position Xout_L=f_(L)(Xin_L) on the GUI region asshown in FIG. 12 using the solid line.

Further, in the third mapping, with regard to the right touch pointXin_R on the operation surface 11, the right touch point Xin_R isassociated with a position Xout_R=f_(f)(Xin_R) on the GUI region asshown in FIG. 12 using a dotted line.

Here, the functions f_(L)(Xin_L) and f_(R)(Xin_R) indicate the thirdmapping.

According to the function f_(L)(Xin_L) in FIG. 12, with regard to theleft touch point Xin_L, if the left touch point Xin_L is moved from theleft to the right in a range from the left end point Xin_min=0 to theright end point in1 of the left operation range, the left correspondencepoint Xout_L on the display screen 21 which corresponds to the lefttouch point Xin_L causes the left GUI region to be moved from the leftto the right.

In addition, if the left touch point Xin_L reaches the point in1 of theright end of the left operation range, the left correspondence pointXout_L which corresponds to the left touch point Xin_L reaches the pointout1 of the right end of the left GUI region.

Thereafter, even if the left touch point Xin_L moves to the furtherright side than the point in1 of the right end of the left operationrange, the left correspondence point Xout_L which corresponds to theleft touch point Xin_L reaches the point out1 of the right end of theleft GUI region does not move while being positioned at the point out1of the right end of the left GUI region.

Meanwhile, according to the function f_(R)(Xin_R), with regard to theright touch point Xin_R, even if the right touch point Xin_R is moved inthe range from the left end of the operation surface 11 to the point in2of the left end of the right operation range, the right correspondencepoint Xout_R which corresponds to the right touch point Xin_R does notmove while being positioned at the point out2 of the left end of theright touch point Xin_R.

If the right touch point Xin_R reaches the point in2 of the left end ofthe right operation range from the left side of the operation surface 11and then a range from the point in2 of the left end of the rightoperation range to the point Xin_max of the right end is moved from theleft to the right, the right correspondence point Xout_R on the displayscreen 21 which corresponds to the right touch point Xin_R causes theright GUI region to be moved from the left to the right.

In addition, if the right touch point Xin_R reaches the point Xin_max ofthe right end of the right operation range, the right correspondencepoint Xout_R which corresponds to the right touch point Xin_R reachesthe point Xout_max of the right end of the right GUI region.

Meanwhile, in the third mapping shown in FIGS. 11 and 12, with regard tothe vertical direction (e.g., the y direction), mapping which is thesame as the uniform mapping is performed on both the left touch pointand the right touch point.

In the third mapping, when the user moves the left touch point Xin_Lfrom the left to the right on the operation surface 11 (e.g., in thecentral direction) and the left correspondence point Xout_L whichcorresponds to the left touch point Xin_L reaches the point out1 of theright end of the left GUI region, the left correspondence point Xout_Ldoes not move from the point out1.

Further, in the third mapping, when the user moves the right touch pointXin_R from the right to the left on the operation surface 11 (e.g., inthe central direction) and the right correspondence point Xout_R whichcorresponds to the right touch point Xin_R reaches the point out2 of theleft end of the right GUI region, the right correspondence point Xout_Rdoes not move from the point out2.

Therefore, according to the third mapping, it is possible to prevent the“Y”, “H”, or “B” key of the OSK which is arranged on the left end of theright GUI region from being erroneously operated when trying to operatethe “T”, “G”, or “V” key of the OSK which is arranged on the right endof the left GUI region using the left touch point. On the contrary, itis possible to prevent the “T”, “G”, or “V” key of the OSK which isarranged on the right end of the left GUI region from being erroneouslyoperated when trying to operate the “Y”, “H”, or “B” key of the OSKwhich is arranged on the left end of the right GUI region using theright touch point.

As a result, the user can point to a part which is at the end portion ofthe left GUI region in addition to the “T”, “G”, and “V” keys of the OSKwith a rough operation using the left hand, and can point to a partwhich is at the end portion of the right GUI region in addition to the“Y”, “H”, and “B” keys of the OSK with a rough operation using the righthand, and thus it is possible to improve the operability.

Here, in the third mapping, it is possible to individually limit aregion which can be pointed to by the left hand and a region which canbe pointed to by the right hand of the GUI region. Hereinafter, thethird mapping is referred to as a limited mapping.

Meanwhile, in FIGS. 11 and 12, as the limited mapping (e.g., the thirdmapping), the mapping is performed to limit the region which can bepointed to using the left hand and the region which can be pointed tousing the right hand of the GUI region in the horizontal direction.However, in addition thereto, it is possible to limit the regions whichcan be respectively pointed to using the left hand and the right hand inthe vertical direction or the like.

In addition, in the limited mapping, it is possible to apply the secondmapping in order to associate the left operation range with the left GUIregion and to associate the right operation range with the right GUIregion.

However, if the limited mapping is regularly applied, there is a case inwhich a trouble occurs in the operation.

That is, for example, if the limited mapping in FIGS. 11 and 12 isregularly applied, when the operation surface 11 is operated using onlyeither the left hand or the right hand, some keys of the OSK might notbe operated by the hand.

Here, the information processing unit 52 of the TV 20 (FIG. 4) performsapplication determination in order to determine whether or not to applythe limited mapping, and can determine whether or not to apply thelimited mapping using the mapping unit 62 according to the result of theapplication determination.

FIG. 13 is a flowchart illustrating a determination process to apply thelimited mapping, in accordance with an embodiment of the presentdisclosure.

In step S31, the information processing unit 52 recognizes the number oftouch points on the operation surface 11 based on the operationinformation which is supplied from the communication unit 51, and theprocess proceeds to step S32.

In step S32, the information processing unit 52 determines whether ornot the number of touch points is two.

In step S32, when it is determined that the number of touch points istwo, that is, when the user touches the operation surface 11 by both(e.g., a finger of) the left hand and (e.g., a finger of) the righthand, and thus the left touch point and the right touch point arepresent, the process proceeds to step S33. The information processingunit 52 determines to perform the limited mapping using the mapping unit62, and the process returns to step S31.

In addition, when it is determined that the number of touch points isnot two in step S33, that is, for example, when the user touches theoperation surface 11 by either the left hand or the right hand, or whenthe user does not touch the operation surface 11, the process proceedsto step S34. The information processing unit 52 determines not toperform (e.g., to release) the limited mapping using the mapping unit62, and the process returns to step S31.

According to the above-described application determination process, themapping of the mapping unit 62 is changed according to the number oftouch points. That is, only when the user touches the operation surface11 by both the left hand and the right hand, is the limited mappingperformed. In the other cases, for example, when the user touches theoperation surface 11 by only either the left hand or the right hand, thelimited mapping is not performed.

Therefore, for example, even if the user operates the operation surface11 by only either the left hand or the right hand, all of the keys ofthe OSK may be operated by the limited mapping.

However, in the limited mapping, when the region which can be pointed tousing the left hand and the region which can be pointed to using theright hand of the GUI region are individually limited, and also when thepointer 30L or 30R is displayed, it is necessary to recognize whether atouch point (or a correspondence point) is a left touch point (or a leftcorrespondence point) touched by the left hand of the user or a righttouch point (or a right correspondence point) touched by the right handof the user.

Here, the information processing unit 52 of the TV 20 (FIG. 4) performsa touch point recognition process of recognizing whether a touch pointrecognized from the operation information which is supplied from thecommunication unit 51 is a left touch point or a right touch point.

FIG. 14 is a flowchart illustrating a touch point recognition process,in accordance with an embodiment of the present disclosure.

In step S41, the information processing unit 52 determines whether ornot the number of touch points on the operation surface 11 has increasedbased on the operation information which is supplied from thecommunication unit 51.

When it is determined that the number of touch points has not increasedin step S41, the process proceeds to step S47.

In addition, when it is determined that the number of touch points hasincreased in step S41, that is, when a new touch point has appeared, theprocess proceeds to step S42. The information processing unit 52determines whether or not the new touch point is a first touch point.

When it is determined that the new touch point is the first touch pointin step S42, the process proceeds to step S43. The informationprocessing unit 52 determines whether or not the x-coordinate X1 of thefirst touch point as the new touch point is less than the x-coordinateXmax/2 of a middle point (hereinafter, referred to as a middle pointcoordinate) in the horizontal direction of the operation surface 11.

When it is determined that the x-coordinate X1 of the first touch pointis less than the middle point coordinate Xmax/2 in step S43, that is,when the first touch point is a point on the left half side of theoperation surface 11, the process proceeds to step S44. The informationprocessing unit 52 recognizes that the first touch point is the lefttouch point.

Thereafter, the information processing unit 52, according to therecognition that the first touch point is the left touch point, setsfirst touch point coordinates (X1,Y1) (e.g., which are recognized fromthe operation information) to (e.g., a variable which indicates) lefttouch point coordinates (Xleft,Yleft), and sets a value (none,none)which indicates that the right touch point is not present (e.g.,undefined) to right touch point coordinates (Xright,Yright). Thereafter,the process returns from step S44 to step S41.

In addition, when it is determined that the x-coordinate X1 of the firsttouch point is not less than the middle point coordinate Xmax/2 in stepS43, that is, when the first touch point is a point on the right halfside of the operation surface 11, the process proceeds to step S45. Theinformation processing unit 52 recognizes that the first touch point isthe right touch point.

Thereafter, the information processing unit 52, according to therecognition that the first touch point is the right touch point, sets avalue (none,none) which indicates that the left touch point is notpresent to the left touch point coordinates (Xleft,Yleft), and sets thefirst touch point coordinates (X1,Y1) to the right touch pointcoordinates (Xright,Yright). Thereafter, the process returns from stepS45 to step S41.

Meanwhile, when it is determined that the new touch point is not thefirst touch point in step S42, the process proceeds to step S46. Theinformation processing unit 52 determines whether or not the new touchpoint is a second touch point.

When it is determined that the new touch point is not the second touchpoint in step S46, the process proceeds to step S47. The informationprocessing unit 52 maintains the recognition of the current left touchpoint and the right touch point as it is, and sets the left touch pointcoordinates (Xleft,Yleft) and the right touch point coordinates(Xright,Yright) according to the recognition. Thereafter, the processreturns to step S41.

In addition, when it is determined that the new touch point is thesecond touch point in step S46, the process proceeds to step S48. Theinformation processing unit 52 determines whether or not thex-coordinate X2 of the second touch point is greater than thex-coordinate X1 of the first touch point.

When it is determined that the x-coordinate X2 of the second touch pointis greater than the x-coordinate X1 of the first touch point in stepS48, that is, when the second touch point is present on the furtherright side than the first touch point on the operation surface 11, theprocess proceeds to step S49. The information processing unit 52recognizes that the first touch point is the left touch point and thesecond touch point is the right touch point.

Thereafter, the information processing unit 52, according to therecognition that the first touch point is the left touch point and thesecond touch point is the right touch point, sets the first touch pointcoordinates (X1,Y1) to the left touch point coordinates (Xleft,Yleft)and sets the second touch point coordinates (X2,Y2) (e.g., which isrecognized from the operation information) to the right touch pointcoordinates (Xright,Yright). The process returns from step S49 to stepS41.

In addition, when it is determined that the x-coordinate X2 of thesecond touch point is not greater than the x-coordinate X1 of the firsttouch point in step S48, that is, when the second touch point is presenton the further left side than the first touch point on the operationsurface 11, the process proceeds to step S50. The information processingunit 52 recognizes that the first touch point is the right touch pointand the second touch point is the left touch point.

Thereafter, the information processing unit 52, according to therecognition that the first touch point is the right touch point and thesecond touch point is the left touch point, sets the second touch pointcoordinates (X2,Y2) to the left touch point coordinates (Xleft,Yleft)and sets the first touch point coordinates (X1,Y1) (e.g., which isrecognized from the operation information) to the right touch pointcoordinates (Xright,Yright). The process returns from step S50 to stepS41.

With regard to an operation system (e.g., a second operation system) inwhich the positions on the operation surface 11 have a one to onecorrespondence with the positions on the display screen 21, the userrepeatedly points to necessary spots of the GUI region and performs thedetermination operation as much as necessary by causing right and lefthand fingers to touch the operation surface 11 in order and by movingthe fingers when necessary while the right and left hand fingers aretouching the operation surface 11.

Therefore, it is possible to recognize whether the touch point is theleft touch point or the right touch point by performing the touch pointrecognition process according to the flowchart in FIG. 14.

Meanwhile, in addition thereto, it is possible to perform the touchpoint recognition based on, for example, the shadow (e.g., shape) of afinger on the operation surface 11.

In addition, it is possible to perform the touch point recognition basedon the approach direction of a finger when the finger is approaching theoperation surface 11.

It is possible to detect the approach direction of a finger based on thechange in the contact area (e.g., shape) between the finger and theoperation surface 11.

In addition, when the sensor 12 can detect the position of a finger onthe operation surface 11 in the z direction, it is possible to detectthe approach direction of the finger by dragging the positions (x,y,z)of the approached finger.

<Fourth Mapping>

FIG. 15 is a view illustrating a fourth mapping which associates theoperation surface 11 with the display screen 21, in accordance with anembodiment of the present disclosure.

In the fourth mapping, different types of mappings are performed on theleft touch point and the right touch point in the same manner as thethird mapping.

However, in the third mapping, a part of the operation surface 11 isassociated with a part of the GUI region with regard to the left touchpoint, and another part of operation surface 11 is associated with aremaining part of the GUI region with regard to the right touch point.However, in the fourth mapping, mapping is performed such that the rangeof the operation surface 11 which is associated with the GUI region withregard to the left touch point deviates from the range of the operationsurface 11 which is associated with the GUI region with regard to theright touch point.

Therefore, in a case in which the fourth mapping is applied, it isnecessary to perform the touch point recognition in order to recognizewhether the touch point is the left touch point or the right touch pointin the same manner as the case to which the third mapping is applied.

In FIG. 15, a range from the left end of the operation surface 11 to aposition which is slightly central rather than the right end thereof,that is, a range which has a narrower horizontal width than theoperation surface 11 is a range which can be operated by a left hand(hereinafter, referred to as a left operable range), and a range fromthe right end of the operation surface 11 to a position which isslightly central rather than the left end thereof, that is, a rangewhich has a narrower horizontal width than the operation surface 11 is arange which can be operated by a right hand (hereinafter, referred to asa right operable range).

In addition, in FIG. 15, the left operable range is uniformly associatedwith the GUI region with regard to the left touch point, and the rightoperable range is uniformly associated with the GUI region with regardto the right touch point.

Therefore, in FIG. 15, the operation surface 11 is associated with theGUI region such that the left operable range which is associated withthe GUI region is shifted to the left side from the right operable rangewith regard to the left touch point, and the right operable range whichis associated with the GUI region is shifted to the right side from theleft operable range with regard to the right touch point.

FIG. 16 is a view illustrating the correspondence relationship betweenthe positions (Xin,Yin) on the operation surface 11 and thecorrespondence points (Xout,Yout) on the display screen 21 according tothe fourth mapping, in accordance with an embodiment of the presentdisclosure.

Meanwhile, it is assumed that the length H2 of the horizontal directionof the GUI region is kx (=H2/H1) times as large as the length H1 of thehorizontal direction of the operation surface 11, and that the length V2of the vertical direction of the GUI region is ky (=V2/V1) times aslarge as the length V1 of the vertical direction of the operationsurface 11. Further, it is assumed that kx=ky=k.

In addition, for brief explanation, it is assumed that the GUI region isa rectangular-shaped region, that lower left points on the operationsurface 11 are points (Xin_min,Yin_min), and that upper right points arepoints (Xin_max,Yin_max). Further, it is assumed that the lower leftpoints on the GUI region are points (Xout_min,Yout_min) and the upperright points therein are points (Xout_max,Yout_max).

For example, if it is assumed that(Xin_min,Yin_min)=(Xout_min,Yout_min)=(0,0) at present, in the uniformmapping, positions (Xin,Yin) on the operation surface 11 are associatedwith positions (k×Xin,k×Yin) on the GUI region, as shown in FIG. 16using a dashed line.

Meanwhile, in FIG. 16, the correspondence relationship of they-coordinate is omitted and only the correspondence relationship of thex-coordinate is shown.

In the fourth mapping, with regard to the left touch point Xin_L on theoperation surface 11 (e.g., the left touch point of which thex-coordinate is Xin_L on the operation surface 11), the left touch pointXin_L is associated with a position Xout_L=f_(L)(Xin_L) on the GUIregion, as shown in FIG. 16 using a solid line.

Further, in the fourth mapping, with regard to the right touch pointXin_R on the operation surface 11, the right touch point Xin_R isassociated with a position Xout_R=f_(R)(Xin_R) on the GUI region, asshown in FIG. 16 using a dotted line.

Here, the functions f_(L)(Xin_L) and f_(R)(Xin_R) indicate the fourthmapping.

According to the function f_(L)(Xin_L) in FIG. 16, with regard to theleft touch point Xin_L, the left touch point Xin_L moves only within theleft operable range of the range from the point Xin_min of the right endof the operation surface 11 to the point Xin_max of the left end, thatis, within the range from the point Xin_min of the left end of the leftoperable range to the point inL (<Xin_max) of the right end.

In addition, in the fourth mapping, the left operable range which hasthe narrower horizontal width than the operation surface 11 is uniformlyassociated with the GUI region (e.g., the coordinates of the left touchpoint in the left operable range have a proportional relationship to thecoordinates of the left correspondence point on the GUI region whichcorresponds to the left touch point) as shown in FIG. 16. Therefore, asthe left touch point Xin_L is positioned on the right side, the leftcorrespondence point Xout_L on the GUI region which corresponds to theleft touch point Xin_L is offset (e.g., shifted) (the offset isindicated using an upward arrow in FIG. 16) to the further rightcompared to the case of the normal matching.

In the same manner, in the fourth mapping, the right operable rangewhich has the narrower horizontal width than the operation surface 11 isuniformly associated with the GUI region as shown in FIG. 16. Therefore,as the right touch point Xin_R is positioned on the left side, the rightcorrespondence point Xout_R on the GUI region which corresponds to theright touch point Xin_R is offset (the offset is indicated using adownward arrow in FIG. 16) to the further left compared to the case ofthe normal matching.

Meanwhile, in the fourth mapping shown in FIGS. 15 and 16, with regardto the vertical direction (e.g., y direction), the same mapping as theuniform mapping is performed on both the left touch point and the righttouch point.

Here, for example, with regard to neighboring keys in the OSK, forexample, the “G” key and the “H” key, when trying to continuouslyoperate the “G” key and the “H” key using a left-hand finger and aright-hand finger, respectively, the left-hand finger interferes (e.g.,contact) with the right-hand finger on the operation surface 11 in thenormal mapping, and thus it is difficult to point to the “H” key usingthe right-hand finger while pointing to the “G” key using the left-handfinger.

In addition, in a case in which the sensor 12 is an electrostatic touchsensor, if the left-hand finger comes into contact with the right-handfinger, there may be a case in which the two touch points of theleft-hand finger and the right-hand finger are detected as a singletouch point which has a large contact area based on an electrostatictouch point detection principle. In this case, against the intention ofthe user, only either the pointers 30L or 30R is displayed instead ofthe two pointers 30L and 30R and it is difficult to rapidly andcontinuously operate the “G” key and the “H” key.

In the fourth mapping, as described above, the left correspondence pointXout_L on the GUI region which corresponds to the left touch point Xin_Lis offset to the further right compared to the normal mapping, and theright correspondence point Xout_R on the GUI region which corresponds tothe right touch point Xin_R is offset to the further left compared tothe normal mapping.

Therefore, in the normal mapping, it is difficult to point to the “G”key unless the left-hand finger is moved such that the left touch pointXin_L is positioned on the left near the center of the operation surface11. However, according to the fourth mapping, it is possible to point tothe “G” key using the left correspondence point Xout_L which is offsetto the right by moving the left-hand finger such that the left touchpoint Xin_L is positioned on the left which is a little away from thecenter of the operation surface 11.

In the same manner, in the normal mapping, it is difficult to point tothe “H” key unless the right-hand finger is moved such that the righttouch point Xin_R is positioned on the right near the center of theoperation surface 11. However, according to the fourth mapping, it ispossible to point to the “H” key using the right correspondence pointXout_R which is offset to the left by moving the right-hand finger suchthat the right touch point Xin_R is positioned on the left which is alittle away from the center of the operation surface 11.

As a result, with regard to the neighboring keys in the OSK, that is,the “G” key and the “H” key, when trying to continuously operate the “G”key and the “H” key using the left-hand finger and the right-handfinger, respectively, it is possible to point to the “G” key using theleft-hand finger and point to the “H” key using the right-hand finger ina state in which the left-hand finger is a little away from theright-hand finger on the operation surface 11.

Therefore, when trying to continuously operate the neighboring keys inthe OSK using the left-hand finger and the right-hand finger, theleft-hand finger and the right-hand finger are prevented frominterfering with each other, and the two touch points of the left-handfinger and the right-hand finger are prevented from being detected as asingle touch point having a large contact area, and thus it is possibleto improve the operability.

Meanwhile, in the fourth mapping shown in FIGS. 15 and 16, the leftoperable range is associated with the whole GUI region and the rightoperable range is associated with the whole GUI region. Therefore, it ispossible to point to an arbitrary point in the GUI region which includesall the keys of the OSK using only either the left-hand finger or theright-hand finger.

Here, in the fourth mapping, based on the fact of whether the touchpoint is the left touch point or the right touch point and thecoordinate of the touch point (e.g., in FIGS. 15 and 16, thex-coordinate), it is possible to perform correction in order to offsetthe correspondence point of the touch point. Hereinafter, the fourthmapping is referred to as an offset mapping.

In the offset mapping in FIGS. 15 and 16, with regard to the left touchpoint, as the x-coordinate Xin_L of the left touch point becomes larger,the amount of offset (indicated by an upward arrow in FIG. 16) in thepositive direction of the x-coordinate Xout_L of the left correspondencepoint increases compared to the case of the uniform mapping (indicatedby a dashed line in FIG. 16) which associates the touch points (Xin,Yin)with the correspondence points (k×Xin,k×Yin).

Therefore, as the left touch point is positioned on the right side, theleft correspondence point is offset to the further right side comparedto the case of the uniform mapping.

In addition, in the offset mapping in FIGS. 15 and 16, with regard tothe right touch point, as the x-coordinate Xin_R of the right touchpoint is small, the amount of offset (indicated by a downward arrow inFIG. 16) in the negative direction of the x-coordinate Xout_R of theright correspondence point increases compared to the case of the uniformmapping (indicated by the dashed line in FIG. 16).

Therefore, as the right touch point is positioned on the left side, theright correspondence point is offset to the further left side comparedto the case of the uniform mapping.

As described above, the left correspondence point is offset to the rightside and the right correspondence point is offset to the left side.Therefore, when the left-side key of the two keys of the OSK which arehorizontally adjacent to each other is operated by the left-hand fingerand the right-side key is operated by the right-hand finger, it ispossible to point to two adjacent keys using the left-hand finger andthe right-hand finger in the state in which the left-hand finger is alittle away from the right-hand finger on the operation surface 11.

Meanwhile, in FIGS. 15 and 16, the mapping which offsets thecorrespondence point in the horizontal direction is performed as theoffset mapping (e.g., fourth mapping). However, in addition thereto, inthe offset mapping, it is possible to offset a correspondence pointwhich is positioned on the upper side between the left correspondencepoint and the right correspondence point to the lower side, and tooffset a correspondence point which is positioned on the lower side tothe upper side.

In this case, for example, when two keys of the OSK which are verticallyadjacent to each other are operated by the left-hand finger and theright-hand finger, it is possible to prevent the left-hand finger andthe right-hand finger from interfering with each other.

In addition, when the left operable range is associated with the GUIregion and the right operable range is associated with the GUI region inthe offset mapping, it is possible to apply the second mapping.

Meanwhile, the interference or the like between the left-hand finger andthe right-hand finger occurs when the operation surface 11 is operatedusing the left-hand finger and the right-hand finger. Therefore, theoffset mapping is not applied when the touch point is a single point orless, and the offset mapping can be applied only when the touch pointsare two points.

However, if the offset mapping is not applied when the touch point is asingle point and the offset mapping is set to be applied only when thetouch points are two points, the pointers 30L and 30R which aredisplayed on the display screen 21 jumps to move when the touch point ischanged from a single point to two points and when two points arechanged into a single point. As a result, the user may be caused to feela sense of incompatibility.

Therefore, it is possible to apply the offset mapping regardless of thenumber of touch points.

Meanwhile, as described in FIG. 4, when the operation surface 11 isintegrally configured with the display unit such as a liquid crystalpanel or the like and functions as a touch screen which can display animage and can receive an input with respect to the image, it is possibleto display, for example, the GUI which is displayed on the displayscreen 21 of the TV 20 and, for example, the OSK which is the GUIcomponent configuring the GUI, on the operation surface 11 whichfunctions as the touch screen in the terminal 10.

For example, when the OSK which is displayed on the display screen 21 ofthe TV 20 is also displayed on the operation surface 11 of the terminal10, there may be cases in which the user operates the operation surface11 while watching the OSK which is displayed on the display screen 21 ofthe TV 20 apart from the user, and in which the user operates theoperation surface 11 while watching the OSK which is displayed on theoperation surface 11 of the terminal 10 on hand.

If the offset mapping is applied when the user operates the operationsurface 11 while watching the OSK which is displayed on the operationsurface 11 of the terminal 10 on hand, there is a case in which the keyof the OSK touched by the user on the operation surface 11 does notcoincide with the key of the OSK pointed to by the pointer 30L or 30R onthe display screen 21. In this case, the user may be caused to feel asense of incompatibility.

Here, the application determination is performed in order to determinewhether or not to apply the offset mapping in the information processingunit 52 of the TV 20 (FIG. 4), and thus it is possible to determinewhether or not to apply the offset mapping in the mapping unit 62according to the result of the application determination.

FIG. 17 is a flowchart illustrating a process of the applicationdetermination of the offset mapping, in accordance with an embodiment ofthe present disclosure.

In step S61, the information processing unit 52 determines whether ornot the operation performed by the user on the operation surface 11 istapping based on the operation information which is supplied from thecommunication unit 51.

In step S61, when it is determined that the operation performed by theuser on the operation surface 11 is tapping, the process proceeds tostep S62. The information processing unit 52 determines that the offsetmapping is not performed (e.g., released) in the mapping unit 62, andthe process returns to step S61.

In addition, when it is determined that the operation performed by theuser on the operation surface 11 is not tapping in step S61, the processproceeds to step S63. The information processing unit 52 determineswhether or not the operation performed by the user on the operationsurface 11 is dragging based on the operation information which issupplied from the communication unit 51.

When it is determined that the operation performed by the user on theoperation surface 11 is not dragging in step S63, the process returns tostep S61.

In addition, when it is determined that the operation performed by theuser on the operation surface 11 is dragging in step S63, the processproceeds to step S64. The information processing unit 52 determines toperform the offset mapping in the mapping unit 62, and the processreturns to step S61.

That is, when the user performs a drag operation on the operationsurface 11, it is assumed that the user operates the operation surface11 while watching the OSK or the like which is displayed on the displayscreen 21 of the TV 20. Therefore, the mapping unit 62 performs theoffset mapping.

On the other hand, when the user performs a tap operation on theoperation surface 11, it is assumed that the user operates the operationsurface 11 while watching the OSK or the like which is displayed on theoperation surface 11. Therefore, the mapping unit 62 does not performthe offset mapping.

In this case, it is possible to prevent the user from feeling a sense ofincompatibility while the key of the OSK touched by the user on theoperation surface 11 does not coincide with the key of the OSK pointedto by the pointer 30L or 30R on the display screen 21.

Meanwhile, when the operation surface 11 functions as the touch screen,the terminal 10 can be operated in a stand-alone mode like a smart phoneor the like.

In this case, the terminal 10 is provided with the same block as themapping unit 62 and the correspondence point detection unit 63 shown inFIG. 4, and can perform mapping and detection of a correspondence pointwhich corresponds to a touch point.

FIG. 18 is a view illustrating an example of the display of theoperation surface 11 as the touch screen of such a terminal 10, inaccordance with an embodiment of the present disclosure.

In FIG. 18, the OSK is displayed on the operation surface 11, and theinput section is displayed on the upper portion of the OSK.

In FIG. 18, letters which are input by operating the OSK are displayedin the input section.

In FIG. 18, the offset mapping which performs correction in order tooffset the correspondence point which corresponds to the touch point isperformed in both directions of the x direction (e.g., the horizontaldirection) and the y direction (e.g., the vertical direction) of theterminal 10.

However, with regard to the x direction, the offset mapping in which theoffset amount of the x-coordinate of the correspondence point whichcorresponds to the touch point varies according to the x-coordinate ofthe touch point (hereinafter, referred to as a variable offset mapping)is performed as described in FIG. 15. However, with regard to the ydirection, the offset mapping in which the offset amount of they-coordinate of the correspondence point which corresponds to the touchpoint is uniform regardless of the y-coordinate of the touch point(hereinafter, referred to as a fixed offset mapping) is performed.

In FIG. 18, in terminal 10, the left correspondence point whichcorresponds to the left touch point and the right correspondence pointwhich corresponds to the right touch point are detected based on theresult of the above-described offset mapping. A pointer 15L is displayedat the left correspondence point of the operation surface 11 as a symbolimage which indicates that the left-hand finger is touched, a pointer15R is displayed at the right correspondence point of the operationsurface 11 as a symbol image which indicates that the right-hand fingeris touched.

In FIG. 18, as described above, the variable offset mapping is performedwith regard to the x direction, and the fixed offset mapping isperformed with regard to the y direction. Therefore, when the left touchpoint is positioned at the left end of the OSK which is displayed on theoperation surface 11 as shown in FIG. 18A, the left correspondence pointwhich corresponds to the left touch point is positioned obtained byoffsetting a predetermined offset amount in the y direction (upperdirection) from the left touch point.

As described above, since the pointer 15L is displayed at the leftcorrespondence point of the position obtained by offsetting thepredetermined offset amount in the y direction from the left touchpoint, it is possible to prevent the pointer 15L from being hidden bythe left-hand finger which touches the left touch point and beinginvisible.

In the same manner, with regard to the right touch point, when the righttouch point is positioned at the right end of the OSK which is displayedon the operation surface 11, the pointer 15R is displayed at the rightcorrespondence point of the position obtained by offsetting thepredetermined offset amount in the y direction from the right touchpoint. Therefore, it is possible to prevent the pointer 15R from beinghidden by the right-hand finger which touches the right touch point.

In addition, when the left touch point is positioned on the left nearthe center of the OSK which is displayed on the operation surface 11shown in FIG. 18B, the left correspondence point which corresponds tothe left touch point is at a position obtained by offsetting apredetermined offset amount in the y direction from the left touch pointand offsetting the offset amount which increases according to thex-coordinate of the left touch point in the x direction (e.g., rightdirection) (e.g., near center).

As described above, the pointer 15L is displayed at the leftcorrespondence point of the position (e.g., the upper left position ofthe left touch point) near the center by offsetting a predeterminedoffset amount in the y direction from the left touch point and byoffsetting the offset amount which increases according to thex-coordinate of the left touch point in the x direction.

In addition, when the left touch point is positioned on the right nearthe center of the OSK which is displayed on the operation surface 11 asshown in FIG. 18B, the right correspondence point which corresponds tothe right touch point is positioned obtained by offsetting apredetermined offset amount in the y direction from the right touchpoint and offsetting the offset amount which increases according to thex-coordinate of the right touch point in the x direction (e.g., leftdirection) (e.g., near center).

As described above, the pointer 15R is displayed at the rightcorrespondence point of the position (e.g., the upper right position ofthe right touch point) near the center by offsetting by a predeterminedoffset amount in the y direction from the right touch point and byoffsetting the offset amount which increases according to thex-coordinate of the right touch point in the x direction.

Therefore, it is possible to prevent the pointers 15L and 15R from beinghidden by fingers, and it is possible to prevent the left-hand fingerand the right-hand finger from being contact with each other when thetwo adjacent keys of the OSK are operated by the left-hand finger andthe right-hand finger.

As described above, for example, in the stand-alone terminal 10, thevariable offset mapping is performed with regard to the x direction andthe fixed offset mapping is performed with regard to the y direction,and thus it is possible to prevent the pointers 15L and 15R which aredisplayed on the operation surface 11 from being hidden by fingers.Therefore, the user easily recognizes the pointed keys of the OSK bywatching the pointers 15L and 15R, and thus it is possible to performoperation without serious burden even when the keys of the OSK aresmall.

Further, according to the offset mapping in the x direction, it ispossible to reduce a distance in which a finger is moved in the xdirection when pointing to the key of the OSK.

Meanwhile, it is possible to display the pointers 15L and 15R such thatthe user can easily distinguish the respective pointers by changing thedisplay formats thereof in the same manner as the pointers 30L and 30R.

In addition, in FIG. 18, it is possible to display the letters of thekeys of the OSK, which are pointed to and on which the determinationoperation described in FIG. 2 is performed, in the input section. Inaddition thereto, it is possible to perform so-called preview display ofthe letters of the keys of the OSK which are pointed to and on which thedetermination operation is not performed.

Further, when the preview display is performed in the input section, thedisplay can be performed using the same display format as each of thepointers 15L and 15R such that the user easily distinguishes the letterof the key pointed to with the left-hand finger from the letter of thekey pointed to with the right-hand finger. In this case, the user canoperate the OSK by performing a so-called blind touch while watching theinput section.

Meanwhile, in FIG. 18, two-letter preview display, that is, the letterof a key pointed to by the left-hand finger (pointer 15L) and the letterof a key pointed to by the right-hand finger (pointer 15R) is performed.However, when two letters are displayed in the preview display, there isa case in which readability is deteriorated, and thus it is hard toread. Here, in the preview display, it is possible to selectivelydisplay one of the letter of the key pointed to by the pointer 15L andthe letter of the key pointed to by the pointer 15R. That is, in thepreview display, for example, it is possible to display only a singleletter of the key pointed to by the pointer of the pointers 15L and 15R,which has a large movement distance from when the previous determinationoperation is performed.

<Description of Computer in Accordance with Embodiments>

Subsequently, the above-described series of processes can be performedusing hardware or can be performed using software. When a series ofprocesses is performed using software, a program which configures thesoftware is installed in a general purpose computer or the like.

Here, FIG. 19 shows a configuration example of a computer in which aprogram in accordance with an embodiment of the present disclosure whichperforms the above-described series of process is installed.

The program can be recorded in a hard disk 105 or a ROM 103 in advancewhich functions as a recording medium which is built in a computer.

Alternatively, the program can be stored (e.g., recorded) in a removablerecording medium 111. Such a removable recording medium 111 can beprovided as a so-called package software. Here, the removable recordingmedium 111 includes, for example, a flexible disk, a Compact Disc ReadOnly Memory (CD-ROM), a Magneto Optical (MO) disk, a Digital VersatileDisc (DVD), a magnetic disc, a semiconductor memory, and the like.

Meanwhile, the program is installed in a computer from theabove-described removable recording medium 111, and besides, the programcan be downloaded onto a computer via a communication network or abroadcasting network and installed in the built-in hard disk 105. Thatis, for example, the program can be wirelessly transmitted to a computerfrom a downloading site via satellites of digital satellitebroadcasting, or can be transmitted to a computer in a wired manner viaa network such as a Local Area Network (LAN) or the Internet.

The computer includes a built-in Central Processing Unit (CPU) 102, andan input/output interface 110 is connected to the CPU 102 via a bus 101.

When a command is input via the input/output interface 110 in such a waythat the input unit 107 is operated by a user, the CPU 102 executes aprogram which is stored in a Read Only Memory (ROM) 103 according to thecommand. Alternatively, the CPU 102 loads a program which is stored inthe hard disk 105 to a Random Access Memory (RAM) 104, and executes theprogram.

Therefore, the CPU 102 performs a process according to theabove-described flowchart or a process which is performed by theconfiguration of the above-described block diagram. In addition, the CPU102, for example, causes the result of the process to be output from anoutput unit 106, to be transmitted from a communication unit 108, or tobe recorded in the hard disk 105 via the input/output interface 110 whennecessary.

Meanwhile, the input unit 107 includes a keyboard, a mouse, amicrophone, and the like. In addition, the output unit 106 includes aLiquid Crystal Display (LCD), a speaker, and the like.

Here, in the present specification, the process which is performed by acomputer according to a program is not necessarily chronologicallyperformed according to the order which is described in the flowchart.That is, the process which is performed by the computer according to theprogram includes processes which are performed in parallel orindividually (for example, a parallel process or a process according toan object).

In addition, the program may be processed by a single computer (e.g.,processor), or may be distributed to be processed by a plurality ofcomputers. Further, the program may be transmitted to a remote computerto be executed.

Further, in the present specification, the system means an aggregate ofa plurality of configuring elements (e.g., apparatuses, modules(components), and the like), and it does not matter whether all in theconfiguring elements are included in the same housing. Therefore, in aplurality of apparatuses which are stored in the individual housings andconnected via the network, and a single apparatus in which a pluralityof modules are stored in a single housing, any one is also a system.

Meanwhile, the embodiments of the present disclosure are not limited tothe above-described embodiments, and various types of modifications arepossible in a range which does not depart from the gist of the presentdisclosure.

For example, embodiments of the present disclosure can use theconfiguration of cloud computing in which a single function is sharedbetween a plurality of apparatuses over a network and jointly executed.

In addition, the respective steps in the above-described flowchart canbe not only shared with and executed in a plurality of apparatuses butalso executed in a single apparatus.

Further, when a plurality of processes configure a single step, theplurality of processes included in the single step can be not onlyshared with and executed in a plurality of apparatuses but also executedin a single apparatus.

In addition, in the embodiments, in the terminal 10, the informationwhich indicates a touch point on the operation surface 11 configures theoperation information and then transmitted to the TV 20. However, in theterminal 10, information which indicates a correspondence pointcorresponding to the touch point on the operation surface 11 may beincluded in the operation information and then transmitted to the TV 20.

When, in the terminal 10, the information which indicates thecorrespondence point corresponding to the touch point on the operationsurface 11 configures the operation information and then transmitted tothe TV 20, it is necessary for the terminal 10 to perform mapping anddetection of the correspondence point corresponding to the touch point.Therefore, the mapping unit 62 and the correspondence point detectionunit 63 (FIG. 4) are not provided in the TV 20 but in the terminal 10.

Further, in the embodiments, the display control unit 61 separatelydisplays the pointer 30L or 30R as a symbol image which indicates thatthe correspondence point corresponding to the touch point is touched,from the GUI which is displayed on the display screen 21. However, thedisplay control unit 61 may display that the touch is performed bychanging a display format, for example, the luminance or color of a GUIcomponent which is displayed at the correspondence point instead ofdisplaying the symbol image at the correspondence point.

Meanwhile, embodiments of the present disclosure may include thefollowing configurations.

(1) An information processing apparatus including:

an operation surface configured to receive touch inputs; and

a sensor unit configured to detect at least one right-side touch inputfrom a manipulation by a user of the operation surface within a firstoperational area of the operation surface, and at least one left-sidetouch input from a manipulation by the user of the operation surfacewithin a second operational area of the operation surface,

wherein the first operational area and the second operational area ofthe operation surface are mapped to a graphical user interface (GUI)area of a display device.

(2) The information processing apparatus of (1), wherein the firstoperational area is spaced apart and separated from the secondoperational area by a gap on the operation surface.

(3) The information processing apparatus of (1), wherein the firstoperational area and the second operational area are at least partiallyoverlapping each other.

(4) The information processing apparatus of (1), wherein the firstoperational area and the second operational area substantially overlapeach other.

(5) The information processing apparatus of (1), wherein, upon detectionof the at least one right-side touch input, a first focus correspondingto the detected at least one right-side touch input is displayed in theGUI area of the display device, and wherein, upon detection of the atleast one left-side touch input, a second focus corresponding to thedetected at least one left-side touch input is displayed in the GUI areaof the display device.

(6) The information processing apparatus of (5), wherein the first focusis not displayed in the GUI area of the display device when the at leastone right-side touch input is no longer detected, and the second focusis not displayed in the GUI area of the display device when the at leastone left-side touch input is no longer detected.

(7) The information processing apparatus of (5), wherein the first focusand the second focus each comprise a pointer icon.

(8) The information processing apparatus of (5), wherein the first focusand the second focus each comprise a highlighting of a respectiveassociated region within the GUI area.

(9) The information processing apparatus of(5), wherein the first focusand the second focus each comprise a highlighting of a respectiveassociated key of an on-screen keyboard (OSK) displayed within the GUIarea of the display device.

(10) The information processing apparatus of (1), wherein, upondetection of a first operation tool approaching the operation surfacefor delivering the at least one right-side touch input, a first focuscorresponding to the detected approaching first operation tool isdisplayed in the GUI area of the display device, and wherein, upondetection of a second operation tool approaching the operation surfacefor delivering the at least one left-side touch input, a second focuscorresponding to the detected approaching second operation tool isdisplayed in the GUI area of the display device.

(11) The information processing apparatus of (1), wherein the sensorunit is further configured to recognize touch inputs received by theoperation surface as either right-hand touch inputs or left-hand touchinputs.

(12) The information processing apparatus of (11), wherein the sensorunit recognizes each of the touch inputs as either a right-hand touchinput or a left-hand touch input based on x-coordinate locations ofpoints at which respective touch inputs are made on the operationsurface.

(13) The information processing apparatus of (11), wherein the sensorunit recognizes each of the touch inputs as either a right-hand touchinput or a left-hand touch input based on a shadow shape made on theoperation surface when the operation surface receives a respective touchinput.

(14) The information processing apparatus of (1), wherein the GUI areaof the display device to which the first operational area and the secondoperational area of the operation surface are mapped is smaller than atotal display area of the display device.

(15) The information processing apparatus of (1), wherein the firstoperational area of the operation surface is mapped to a first area ofthe GUI area of the display device, and the second operational area ofthe operation surface is mapped to a second area of the GUI area of thedisplay device.

(16) The information processing apparatus of (15), wherein theright-operational area of the operation surface is mapped to a firstsection of an on-screen keyboard (OSK) of the GUI area of the displaydevice, and the left-operational area of the operation surface is mappedto a second section of the OSK of the GUI area of the display device.

(17) The information processing apparatus of (1), wherein a second GUIis overlaid on an area of the operation surface, the second GUIcorresponding to the GUI area of the display device.

(18) The information processing apparatus of (1), wherein a text-entryportion of the operation surface is mapped to an on-screen keyboard(OSK) of the GUI area of the display device.

(19) The information processing apparatus of (18), wherein a mappingsize ratio of mapping the text-entry portion of the operation surface tothe OSK of the GUI area of the display device is larger than a mappingsize ratio of other portions of the operation surface mapped tocorresponding portions of the GUI area of the display device.

(20) An information processing method including:

mapping a first operational area and a second operational area of anoperation surface of a terminal device to a graphical user interface(GUI) area of a display device;

detecting at least one right-side touch input from a manipulation by auser of the operation surface within the first operational area of theoperation surface; and

detecting at least one left-side touch input from a manipulation by theuser of the operation surface within the second operational area of theoperation surface.

(21) The information processing method of (20), further including:

displaying, in the GUI area of the display device, a first focuscorresponding to the detected at least one first touch input when the atleast one first touch input is detected; and

displaying, in the GUI area of the display device, a second focuscorresponding to the detected at least one second touch input when theat least one second touch input is detected.

(22) A non-transitory computer-readable medium embodied with a program,which when executed by a computer, causes the computer to perform amethod including:

mapping a first operational area and a second operational area of anoperation surface of a terminal device to a graphical user interface(GUI) area of a display device;

detecting at least one right-side touch input from a manipulation by auser of the operation surface within the first operational area of theoperation surface; and

detecting at least one left-side touch input from a manipulation by theuser of the operation surface within the second operational area of theoperation surface.

(23) The non-transitory computer-readable medium of (22), wherein thecomputer is caused to further perform:

displaying, in the GUI area of the display device, a first focuscorresponding to the detected at least one first touch input when the atleast one first touch input is detected; and

displaying, in the GUI area of the display device, a second focuscorresponding to the detected at least one second touch input when theat least one second touch input is detected.

(24) An information processing apparatus including: a mapping unit thatperforms a mapping which associates a position on an operation surfacewhich is operated by a user with a position on a display screen on whichan image is displayed based on graphics which are displayed on thedisplay screen; and a detection unit that detects a correspondence pointwhich is a position on the display screen and corresponds to theposition on the operation surface which is operated by the user based ona result of the mapping.

(25) The information processing apparatus of (24) further including: adisplay control unit that displays a symbol image which indicates thatan operation is performed at the correspondence point or that changes adisplay format of graphic components which configure the graphics on thecorrespondence point.

(26) In the information processing apparatus of (24) or (25), themapping unit performs the mapping based on the graphics which aredisplayed on the display screen such that the position on the operationsurface is associated with a position on a graphic region on which thegraphics are displayed within the position on the display screen.

(27) In the information processing apparatus of (26), the mapping unitperforms the mapping such that the operation surface is uniformlyassociated with the graphic region.

(28) In the information processing apparatus of any one of (24) to (26),the mapping unit performs the mapping based on the graphic componentswhich configure the graphics displayed on the display screen such thatthe position on the operation surface is associated with the position onthe display screen.

(29) In the information processing apparatus of (28), the mapping unitperforms the mapping such that a region on which small-sized graphiccomponents are displayed is associated with a wider range of theoperation surface, compared to a uniform mapping which uniformlyassociates the operation surface with the graphic region.

(30) In the information processing apparatus of any one of (26) to (29),the mapping unit performs different types of mapping with regard to anoperation which is performed on the operation surface by a left hand ofa user and an operation which is performed on the operation surface by aright hand.

(31) In the information processing apparatus of (30), the mapping unitperforms an offset mapping which causes the range of the operationsurface which is associated with the graphic region with regard to theoperation performed on the operation surface by a left hand of the userto be shifted, from the range of the operation surface which isassociated with the graphic region with regard to the operation on theoperation surface by the right hand.

(32) In the information processing apparatus of (31), in the offsetmapping, the mapping unit causes the range of the operation surfacewhich is associated with the graphic region to be shifted to a left sidewith regard to the operation which is performed on the operation surfaceby the left hand of the user, and causes the range of the operationsurface which is associated with the graphic region to be shifted to aright side with regard to the operation which is performed on theoperation surface by the right hand.

(33) In the information processing apparatus of (31), the mapping unitadopts the offset mapping when a drag operation is performed on theoperation surface, and does not adopt the offset mapping when a tapoperation is performed on the operation surface.

(34) In the information processing apparatus of (30), the mapping unitperforms the mapping such that a part of the operation surface isassociated with a part of the graphic region with regard to theoperation which is performed on the operation surface by the left handof the user, and that another part of the operation surface isassociated with a remaining part of the graphic region with regard tothe operation which is performed on the operation surface by the righthand of the user.

(35) In the information processing apparatus of (34), the mapping unitperforms the mapping such that a part of the left side of the operationsurface is associated with a remaining left half of the graphic regionwith regard to the operation which is performed on the operation surfaceby the left hand of the user, and that a part of a right side of theoperation surface is associated with a remaining right half of thegraphic region with regard to the operation which is performed on theoperation surface by the right hand of the user.

(36) In the information processing apparatus of any one of (24) to (35),the mapping unit changes the mapping according to a number of positionson the operation surface which are operated by the user.

(37) An information processing method including: performing a mappingwhich associates a position on an operation surface which is operated bya user with a position on a display screen on which an image isdisplayed based on graphics which are displayed on the display screen;and detecting a correspondence point which is a position on the displayscreen and corresponds to the position on the operation surface which isoperated by the user based on a result of the mapping.

(38) A program causing a computer to function as: a mapping unit thatperforms a mapping which associates a position on an operation surfacewhich is operated by a user with a position on a display screen on whichan image is displayed based on graphics which are displayed on thedisplay screen; and a detection unit that detects a correspondence pointwhich is a position on the display screen and corresponds to theposition on the operation surface which is operated by the user based ona result of the mapping.

(39) An information processing system including: a terminal thatincludes an operation surface which is operated by a user; and aninformation processing apparatus which performs a process according toan operation of the operation surface. The information processingapparatus includes a mapping unit that performs a mapping whichassociates a position on the operation surface with a position on adisplay screen on which an image is displayed based on graphics whichare displayed on the display screen; and a detection unit that detects acorrespondence point which is a position on the display screen andcorresponds to the position on the operation surface which is operatedby the user based on a result of the mapping.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1-20. (canceled)
 21. An information processing apparatus comprising: asensor unit configured to detect at least one left-side touch input by auser within a first operational area which is mapped to a left side areaof an on-screen keyboard (OSK) of a display device, and at least oneright-side touch input by the user within a second operational areawhich is mapped to a right side area of the OSK of the display device;and an output unit configured to output an information for displaying aleft pointer based on a left correspondence point which corresponds tothe left-side touch input and a right pointer based on a rightcorrespondence point which corresponds to the right-side touch input,wherein there is a gap which is not mapped to the OSK between the firstoperational area and the second operational area.
 22. The informationprocessing apparatus of claim 21, wherein when the user moves theleft-side touch input from left to right on the first operational areaand the left correspondence point reaches a point on a right edge of theleft side area of OSK, the left correspondence point does not move fromthe reached point on the right edge with further left to right movementof the left-side touch input.
 23. The information processing apparatusof claim 22, wherein when the user moves the right-side touch input fromright to left on the second operational area and the rightcorrespondence point reaches a point on a left edge of the right sidearea of OSK, the right correspondence point does not move from thereached point on the left edge with further right to left movement ofthe right-side touch input.
 24. The information processing apparatus ofclaim 21, wherein when the user moves the right-side touch input fromright to left on the second operational area and the rightcorrespondence point reaches a point on a left edge of the right sidearea of OSK, the right correspondence point does not move from thereached point on the left edge with further right to left movement ofthe right-side touch input.
 25. The information processing apparatus ofclaim 21, wherein when the user moves the left-side touch input fromleft to right on the first operational area and the left correspondencepoint reaches a right edge of the left side area of OSK, the leftcorrespondence point does not move further beyond the right edge. 26.The information processing apparatus of claim 25, wherein when the usermoves the right-side touch input from right to left on the secondoperational area and the right correspondence point reaches a left edgeof the right side area of OSK, the right correspondence point does notmove further beyond the left edge.
 27. The information processingapparatus of claim 21, wherein when the user moves the right-side touchinput from right to left on the second operational area and the rightcorrespondence point reaches a left edge of the right side area of OSK,the right correspondence point does not move further beyond the leftedge.
 28. The information processing apparatus of claim 21, wherein apart of the first operational area and a part of the second operationalarea are both mapped to a same key on the OSK.
 29. The informationprocessing apparatus of claim 28, wherein the same key is a space key.30. The information processing apparatus of claim 21, furthercomprising: a click switch which detects a pressing operation.
 31. Aninformation processing method comprising: detecting at least oneleft-side touch input by a user within a first operational area which ismapped to a left side area of an on-screen keyboard (OSK) of a displaydevice; detecting at least one right-side touch input by the user withina second operational area which is mapped to a right side area of theOSK of the display device; outputting an information for displaying aleft pointer based on a left correspondence point which corresponds tothe left-side touch input; and outputting a right pointer based on aright correspondence point which corresponds to the right-side touchinput, wherein there is a gap which is not mapped to the OSK between thefirst operational area and the second operational area.
 32. Anon-transitory computer-readable medium having embodied thereon aprogram, which when executed by a computer causes the computer toexecute a method, the method comprising: detecting at least oneleft-side touch input by a user within a first operational area which ismapped to a left side area of an on-screen keyboard (OSK) of a displaydevice; detecting at least one right-side touch input by the user withina second operational area which is mapped to a right side area of theOSK of the display device; outputting an information for displaying aleft pointer based on a left correspondence point which corresponds tothe left-side touch input; and outputting a right pointer based on aright correspondence point which corresponds to the right-side touchinput, wherein there is a gap which is not mapped to the OSK between thefirst operational area and the second operational area.